Uniformization for nonhomogeneous Markov chains

Inexact uniformization and GMRES methods for large Markov chains

Inexact algorithms allow certain operations (typically matrix–vector products or certain function evaluations) to be performed inexactly, either out of necessity or deliberately, in view of trading accuracy for speed. We review recent findings that show the impact of the inexact approach in the uniformization and generalized minimal residual (GMRES) methods when computing the transient, station...

Occupation laws for some time-nonhomogeneous Markov chains

We consider finite-state time-nonhomogeneous Markov chains whose transition matrix at time n is I + G/nζ where G is a “generator” matrix, that is G(i, j) > 0 for i, j distinct, and G(i, i) = − ∑ k 6=iG(i, k), and ζ > 0 is a strength parameter. In these chains, as time grows, the positions are less and less likely to change, and so form simple models of age-dependent time-reinforcing schemes. Th...

Large Deviations for a Class of Nonhomogeneous Markov Chains

Large deviation results are given for a class of perturbed non-homogeneous Markov chains on finite state space which formally includes some stochastic optimization algorithms. Specifically, let {Pn} be a sequence of transition matrices on a finite state space which converge to a limit transition matrix P. Let {Xn} be the associated non-homogeneous Markov chain where Pn controls movement from ti...

Perfect Sampling for Nonhomogeneous Markov Chains and Hidden Markov Models By

SERIE RESEHRCH mEmORBnDH APPROXIMATE UNIFORMIZATION FOR CONTINUOUS-TIME MARKOV CHAINS WITH AN APPLICATION TO PERFORMABILITY ANALYSIS

An approximate version of the Standard uniformization technique is introduced for application to continuous-time Markov chains with unbounded jump rates. This technique is shown to be asymptotically exact and an error bound for the order of its accuracy is provided. An illustrative queueing application is included.